Methods and devices for selective flash illumination

ABSTRACT

The various embodiments described herein include methods and/or devices for directing flash illumination. In one aspect, a method is performed at a device including a camera, one or more processors, and memory storing one or more programs configured for execution by the one or more processors. The method includes directing flash illumination to an identified area of interest in a scene by adjusting power supplied to respective elements of a flash array. The method further includes capturing, via the camera, an image of the scene as illuminated by the flash illumination.

TECHNICAL FIELD

The disclosed embodiments relate generally to flash techniques,including but not limited to directing flash illumination utilizing aflash array.

BACKGROUND

Many devices currently employ a flash to help illuminate a scene forcapture by a camera or other optical sensor. However, there is anongoing challenge to use precise and efficient flash devices.Conventional methods for flash generation are imprecise and/orinefficient. For example, many conventional flash devices directillumination toward the center of a scene, regardless of whether themain image target is at that location. In addition, many flash devicesconsume power to direct illumination toward parts of the scene that aretoo distant from the device to be properly illuminated by the flashillumination.

SUMMARY

Accordingly, there is a need for devices with more precise and/orefficient methods for flash illumination. Such methods and devicesoptionally complement or replace conventional flash illumination methodsand devices.

In one aspect, some embodiments include a method performed at a deviceincluding a camera, one or more processors, and memory storing one ormore programs configured for execution by the one or more processors.The method includes directing flash illumination to an identified areaof interest in a scene by adjusting power supplied to respectiveelements of a flash array. The method further includes capturing, viathe camera, an image of the scene as illuminated by the flashillumination.

In another aspect, some embodiments include a device (e.g., a mobiledevice) with a flash array including a plurality of light sources, and acontroller to adjust power supplied to respective light sources of theplurality of light sources. The device is configured to perform themethod described above.

In yet another aspect, some embodiments include a non-transitorycomputer-readable storage medium that has stored therein instructionsthat, when executed by a device, cause the device to perform theoperations of the method described above. In still another aspect, someembodiments include a device with one or more processors; memory storingone or more programs configured to be executed by the one or moreprocessors; and means for performing the method described above.

Thus, devices are provided with more precise/efficient methods for flashillumination, thereby increasing the accuracy, precision, efficiency,and user satisfaction with such devices. Such methods may complement orreplace conventional methods for flash illumination.

BRIEF DESCRIPTION OF THE DRAWINGS

For a better understanding of the various described embodiments,reference should be made to the Description of Embodiments below, inconjunction with the following drawings. The appended drawings, however,merely illustrate the more pertinent features of the present disclosureand are therefore not to be considered limiting, for the description mayadmit to other effective features. Like reference numerals refer tocorresponding parts throughout the figures and description.

FIG. 1A is a diagram illustrating an exemplary implementation of a depthmapping device, in accordance with some embodiments.

FIG. 1B is a diagram illustrating an exemplary implementation of a flashoptimization device, in accordance with some embodiments.

FIG. 2 is a block diagram illustrating an exemplary implementation ofthe flash optimization device, in accordance with some embodiments.

FIGS. 3A-3B illustrate exemplary procedures for projecting structuredlight on a scene, in accordance with some embodiments.

FIGS. 4A-4D illustrate exemplary procedures for projecting structuredlight on areas of interest in a scene, in accordance with someembodiments.

FIGS. 5A-5B illustrate additional exemplary procedures for projectingstructured light on an area of interest in a scene, in accordance withsome embodiments.

FIGS. 6A-6B are flow diagrams illustrating an exemplary method of depthmapping, in accordance with some embodiments.

FIGS. 7A-7D are diagrams illustrating exemplary implementations of aflash array.

FIG. 8 illustrates exemplary flash optimization for directing flashillumination toward areas of interest in a scene.

FIGS. 9A-9C illustrate exemplary flash optimization for directing flashillumination toward a moving area of interest in a scene.

FIGS. 10A-10B are flow diagrams illustrating an exemplary method offlash optimization, in accordance with some embodiments.

In accordance with common practice the various features illustrated inthe drawings may not be drawn to scale. Accordingly, the dimensions ofthe various features may be arbitrarily expanded or reduced for clarity.In addition, some of the drawings may not depict all of the componentsof an apparatus, device, or medium.

DESCRIPTION OF EMBODIMENTS

Reference will now be made to embodiments, examples of which areillustrated in the accompanying drawings. In the following description,numerous specific details are set forth in order to provide anunderstanding of the various described embodiments. However, it will beapparent to one of ordinary skill in the art that the various describedembodiments may be practiced without these specific details. In someinstances, well-known methods, procedures, components, circuits, andnetworks have not been described in detail so as not to unnecessarilyobscure aspects of the embodiments.

It will also be understood that, although the terms first, second, etc.are, in some instances, used herein to describe various elements, theseelements should not be limited by these terms. These terms are used onlyto distinguish one element from another. For example, a first area ofinterest could be termed a second area of interest, and, similarly, asecond area of interest could be termed a first area of interest,without departing from the scope of the various described embodiments.The first area of interest and the second area of interest are bothareas of interest, but they are not the same area of interest.

The terminology used in the description of the various embodimentsdescribed herein is for the purpose of describing particular embodimentsonly and is not intended to be limiting. As used in the description ofthe various described embodiments and the appended claims, the singularforms “a,” “an,” and “the” are intended to include the plural forms aswell, unless the context clearly indicates otherwise. It will also beunderstood that the term “and/or” as used herein refers to andencompasses any and all possible combinations of one or more of theassociated listed items. It will be further understood that the terms“includes,” “including,” “comprises,” and/or “comprising,” when used inthis specification, specify the presence of stated features, integers,steps, operations, elements, and/or components, but do not preclude thepresence or addition of one or more other features, integers, steps,operations, elements, components, and/or groups thereof.

As used herein, the term “if” is, optionally, construed to mean “when”or “upon” or “in response to determining” or “in response to detecting”or “in accordance with a determination that,” depending on the context.Similarly, the phrase “if it is determined” or “if [a stated conditionor event] is detected” is, optionally, construed to mean “upondetermining” or “in response to determining” or “upon detecting [thestated condition or event]” or “in response to detecting [the statedcondition or event]” or “in accordance with a determination that [astated condition or event] is detected,” depending on the context. Inaddition, the term “exemplary” is used in the sense of “serving as anexample, instance, or illustration” and not in the sense of“representing the best of its kind.”

Embodiments of apparatuses and associated processes for using suchapparatuses are described. In some embodiments, the apparatus is aportable electronic device (e.g., communications device), such as amobile telephone or personal digital assistant (PDA), that also containsother functions, such as camera functions. Other portable electronicdevices, such as laptops or tablet computers are optionally used. Itshould also be understood that, in some embodiments, the apparatus isnot a portable communications device, but is another type of device,such as a desktop computer, or an apparatus including a network ofcomponents.

FIG. 1A is a diagram illustrating an exemplary implementation of depthmapping device 100 (sometimes called “device 100”), in accordance withsome embodiments. Device 100 includes camera 102 and projector 104. FIG.1A also shows detection area 106 for device 100. Detection area 106 isthe area in which the field of view of camera 102 and the field ofprojection of projector 104 overlap. In some instances, the horizontalrange of detection area 106 is limited by the range of projector 104. Insome instances, the horizontal range of detection area 106 is limited bythe light-capturing capabilities of camera 102. Device 100 optionallyincludes one or more additional components such as a display, a userinterface, and a flash. In some embodiments, device 100 is a mobiledevice such as a mobile telephone or a tablet computer. In someembodiments, device 100 is a stationary device such as a desktopcomputer or a computer system.

Camera 102 (also sometimes called an “optical sensor”) captures lightfrom the environment, projected through one or more lenses, and convertsthe light to data representing an image. In conjunction with an imagingmodule (e.g., imaging module 234, FIG. 2), camera 102 captures stillimages and/or video. Camera 102 optionally includes charge-coupleddevice (CCD) or complementary metal-oxide semiconductor (CMOS)phototransistors. In some embodiments, camera 102 is capable ofcapturing infrared light and/or near infrared light. In someembodiments, camera 102 is capable of capturing visible and infraredlight.

In conjunction with a projection module (e.g., projection module 230,FIG. 2), projector 104 projects light (e.g., structured light). In someembodiments, projector 104 is a pico projector (also sometimes called a“pocket projector”, “mobile projector”, or “mini beamer”). In someembodiments, projector 104 is capable of projecting infrared lightand/or near infrared light. In some embodiments, projector 104 includesone or more lasers. In some embodiments, projector 104 includes one ormore light-emitting diodes (LEDs). In some embodiments, projector 104includes one or more organic light-emitting diodes (OLEDs). In someembodiments, projector 104 includes one or moreliquid-crystal-on-silicon (LCoS) devices.

FIG. 1B is a diagram illustrating an exemplary implementation of depthmapping and flash illumination device 150 (sometimes called “device150”), in accordance with some embodiments. In some embodiments, device150 includes device 100. Device 150 includes flash 108 as well as camera102 and projector 104. Alternatively, a device may be used for flashillumination but not depth mapping, and thus may include flash 108 andcamera 102 but not projector 104. FIG. 1B also shows illuminated capturearea 110 for device 150. Illuminated capture area 110 is the area ofoverlap for the field of view of camera 102 and the field ofillumination (i.e., the area that can be illuminated) for flash 108. Insome embodiments, projector 104 includes flash 108. In some embodiments,the light-producing elements of projector 104 are used to illuminate thescene

In conjunction with a flash module (e.g., flash module 248, FIG. 2),flash 108 produces directed light to help illuminate a scene (e.g., forcapture by camera 102). In some embodiments, flash 108 is operated inconjunction with camera 102 to illuminate areas of interest in a scenefor capture by camera 102. In some embodiments, flash 108 provides aburst of directed light (e.g., for use in capturing still images) whilein other embodiments flash 108 provides continuously directed light(e.g., for use in capturing video images). As will be discussed furtherwith reference to FIGS. 7A-7D, in some embodiments, flash 108 includesan array of flash elements (e.g., as shown in FIGS. 7A-7C) such as anarray of LED elements (e.g., as shown in FIG. 7C).

FIG. 2 is a block diagram illustrating an exemplary implementation ofdevice 150, in accordance with some embodiments. Device 150, in theexample of FIG. 2, includes one or more processing units (processors orcores) 202, one or more network or other communications interfaces 204,memory 206, and one or more communication buses 208 for interconnectingthese components. Communication buses 208 optionally include circuitry(sometimes called a chipset) that interconnects and controlscommunications between system components. Device 150 also includes auser interface 210. User interface 210 includes display 212. Device 150further includes camera 102, projector 104, and flash 108. In someembodiments, projector 104 is a separate and distinct device from device150. The user interface 210, projector 104, camera 102, and flash 108thus may also connect to the one or more communication buses 208. Insome embodiments, the flash 108 includes a flash array 252 that includesa plurality of light sources and a controller 254 (e.g., a processor,such as a microcontroller) that adjusts power supplied to respectivelight sources of the plurality of light sources. The controller mayalternatively be separate from the flash 108 (e.g., may be one of theone or more processors 202).

In some embodiments, the device 150 includes inputs such as a keyboard,mouse, and/or other input buttons. Alternatively, or in addition, insome embodiments, display 212 includes a touch-sensitive surface, inwhich case display 212 is a touch-sensitive display. In devices thathave a touch-sensitive display, a physical keyboard is optional (e.g., asoft keyboard may be displayed when keyboard entry is needed). Userinterface 210 optionally also includes an audio output device, such asspeakers or an audio output connection connected to speakers, earphones,or headphones. Furthermore, some devices 150 use a microphone and voicerecognition to supplement or replace the keyboard. Optionally, device150 includes an audio input device (e.g., a microphone) to capture audio(e.g., speech from a user). Optionally, device 150 includes a locationdetection component, such as a GPS (global positioning satellite) orother geo-location receiver, for determining the location of device 150.

Memory 206 includes high-speed random-access memory, such as DRAM, SRAM,DDR RAM or other random-access solid-state memory devices; and mayinclude non-volatile memory, such as one or more magnetic disk storagedevices, optical disk storage devices, flash memory devices, or othernon-volatile solid-state storage devices. Memory 206 optionally includesone or more storage devices remotely located from processor(s) 202.Memory 206, or alternately the non-volatile memory device(s) withinmemory 206, includes a non-transitory computer-readable storage medium.In some embodiments, memory 206 or the computer-readable storage mediumof memory 206 stores the following programs, modules, and datastructures, or a subset or superset thereof:

-   -   operating system 226 that includes procedures for handling        various basic system services and for performing hardware        dependent tasks;    -   network communication module 228 for connecting device 150 to        other devices via communication network interface(s) 204 (wired        or wireless) and one or more communication networks, such as the        Internet, cellular telephone networks, mobile data networks,        other wide area networks, local area networks, metropolitan area        networks, and so on;    -   projection module 230 for projecting light (e.g.,        structured-light) in conjunction with projector 104;    -   pattern module 232 for obtaining (e.g., receiving or generating)        structured-light patterns for use with projector 104;    -   imaging module 234 for capturing still and/or video images of a        scene in conjunction with camera 102;    -   depth module 236 for generating and analyzing depth maps,        including the following sub-modules, or a subset or superset        thereof:        -   edge detection sub-module 240 for detecting depth disrupting            edges (e.g., object edges) in the scene;        -   area of interest sub-module 241 for identifying areas of            interest in the scene; and        -   depth mapping sub-module 242 for generating depth maps of            the scene or portions thereof;    -   focus module 238 for focusing camera 102;    -   object recognition module 244 for identifying objects in a        scene;    -   3-D module 246 for recognizing 3-D gestures that utilize device        150;    -   flash module 248 for producing directed light in conjunction        with flash 108, including power supply sub-module 249 for        supplying power to flash 108; and    -   motion prediction module 250 for predicting movement of an        identified object in a scene.

Each of the above identified modules and applications correspond to aset of executable instructions for performing one or more functions asdescribed above and/or in the methods described herein (e.g., thecomputer-implemented methods and other information processing methodsdescribed herein). These modules (i.e., sets of instructions) need notbe implemented as separate software programs, procedures, or modules,and thus various subsets of these modules are, optionally, combined orotherwise re-arranged in various embodiments. In some embodiments,memory 206 stores a subset of the modules and data structures identifiedabove. Furthermore, memory 206 optionally stores additional modules anddata structures not described above. For example, in some embodiments,area of interest sub-module 241 is separate from depth module 236. Insome embodiments, focus module 238 includes an area of interestsub-module for identifying areas of interest in a scene for use withflash 108. In some embodiments, power supply sub-module 249 is separatefrom flash module 248.

In some embodiments, memory 206 also includes one or more applicationmodules. Examples of application modules that are, optionally, stored inmemory 206 include word processing applications, image editingapplications, drawing applications, presentation applications, browserapplications, JAVA-enabled applications, encryption, digital rightsmanagement, voice recognition, and voice replication.

Although FIG. 2 shows an example of device 150, FIG. 2 is intended moreas a functional description of the various features which may be presentin a device than as a structural schematic of the embodiments describedherein. In practice, and as recognized by those of ordinary skill in theart, items shown separately could be combined and some items could beseparated.

Attention is now directed to depth mapping, embodiments of which aredescribed with respect to FIGS. 3A-6B.

FIGS. 3A-3B illustrate exemplary procedures for projecting structuredlight on a scene, in accordance with some embodiments. FIG. 3A showsscene 300 including hand 302 at a first general distance, lamp 304 at asecond general distance (further than the first general distance), andcouch 306 at a third general distance (further than the second generaldistance). (These distances are referred to as general because each ofthese objects is three-dimensional and thus has some variation in itsdepth.) FIG. 3B shows scene 300 including projected structured-lightpattern 310, a dot pattern. FIG. 3B further shows the dots from pattern310 larger and denser at closer distances. Pattern 310 is projected by aprojector (e.g., projector 104, FIG. 1). In some embodiments, thestructured-light patterns applied to a scene include dot patterns, linepatterns, grid patterns, and/or other patterns incorporating structuredlight of various shapes.

FIGS. 4A-4D illustrate exemplary procedures for projecting structuredlight on areas of interest in scene 300, in accordance with someembodiments. FIG. 4A shows scene 300 with identified areas of interest402 (e.g., area of interest 402-1 and 402-2). FIGS. 4B-4D show anenlarged view of area of interest 402-1. FIG. 4B shows area of interest402-1 without a structured-light pattern projected on it. FIG. 4C showsarea of interest 402-1 with projected structured-light pattern 404, aline pattern with parallel lines. FIG. 4C also shows the lines ofpattern 404 effectively disappear in the far field, the far field beingthe part of couch 306 behind hand 302 in this instance. The lines ofpattern 404 effectively disappear in the far field due to differentspatial frequencies for the projected lines of pattern 404 on eachrespective object (e.g., hand 302 and couch 306). The lines havedifferent spatial frequencies due to the differences in distance betweeneach respective object. FIG. 4D shows area of interest 402-1 withprojected structured-light pattern 406, a second line pattern. Pattern406 is a line pattern rotated with respect to pattern 404 (i.e., thelines of pattern 406 are rotated with respect to the lines of pattern404). FIG. 4D also shows the lines of pattern 406 effectively disappearin the far field. In some embodiments, an apparatus calculates thedepths of features within an area of interest based on the distortion inthe projected pattern (e.g., the gradation and/or curve in the projectedlines or the angle between portions of a respective line at an edge). Inaccordance with some embodiments, an apparatus dynamically generatespatterns of structured-light based on the features of objects within aparticular area of interest (e.g., generates line patterns in whichrespective lines intersect an object's edge at a specified angle).

FIGS. 5A-5B illustrate additional exemplary procedures for projectingstructured light on area of interest 402-2 in scene 300, in accordancewith some embodiments. FIGS. 5A-5B show an enlarged view of area ofinterest 402-2. FIG. 5B shows area of interest 402-2 with projectedstructured-light patterns 502. Pattern 502-1 is a line pattern withparallel lines. Pattern 502-2 is a grid pattern, which may be considereda type of line pattern with parallel and perpendicular lines arranged ina grid. Pattern 502-1 is projected along the edges of lamp 304 andpattern 502-2 is projected along the edges of a pillow on couch 306. Inaccordance with some embodiments, pattern 502-1 and pattern 502-2 areparts of a single pattern. Thus, in accordance with some embodiments, apattern includes lines that are generated to be perpendicular tomultiple identified depth disrupting edges within the area of interest(e.g., lines at various orientations relative to one another).

FIGS. 6A-6B are flow diagrams illustrating method 600 of depth mapping,in accordance with some embodiments. In some embodiments, method 600 isgoverned by instructions that are stored in a non-transitorycomputer-readable storage medium (e.g., in memory 206, FIG. 2) and thatare executed by one or more processors of an apparatus (e.g.,processor(s) 202 of device 150, FIG. 2). In some embodiments, method 600is performed at an apparatus comprising a projector, a camera, one ormore processors, and memory storing one or more programs for executionby one or more processors. As described below, method 600 provides a wayto efficiently generate more precise depth maps.

In some embodiments, the apparatus generates (602) a first depth mapthat indicates the variation of depth in a scene at a first resolution(e.g., using depth module 236, FIG. 2). In some embodiments, the firstdepth map is generated using a structured-light pattern. For example,the first depth map is generated using a dot pattern (e.g., pattern 310as shown in FIG. 3B). In some embodiments, the structured-light patternincludes dots, lines, and/or other shapes of structured light.

In some embodiments, to generate the first depth map the apparatus uses(604) a projector (e.g., projector 104, FIG. 2) to apply a firststructured-light pattern (e.g., pattern 310, FIG. 3B) to the scene. Insome embodiments, the first structured-light pattern is distinct fromstructured-light patterns subsequently applied to particular areas ofinterest within the scene. For example, pattern 310 in FIG. 3B isdistinct from pattern 404 in FIG. 4C and pattern 406 in FIG. 4D as wellas patterns 502 in FIG. 5B. In some embodiments, the projector is a picoprojector.

In some embodiments, the apparatus calculates (606) depths in accordancewith a spatial frequency of dots of the first structured-light patternin the scene (e.g., using depth mapping sub-module 242, FIG. 2). Thus,in some embodiments, the first structured-light pattern includes a dotpattern and generating the first depth map includes calculating depthsin accordance with a spatial frequency of dots in the scene. Forexample, the apparatus calculates depths in accordance with the spatialfrequency of dots in pattern 310 as shown in FIG. 3B.

In some embodiments, the first structured-light pattern includes a linepattern.

In some embodiments, the first depth map is obtained from an externalsource, distinct from the apparatus (e.g., obtained via communicationinterface(s) 204, FIG. 2). In some embodiments, the first depth map isgenerated using stereo cameras. For example, each camera generates a 2-Dframe and the frames are combined to create a 3-D image using a knowndistance between the cameras and differences in object positions betweenthe two frames. In some embodiments, the first depth map is generatedusing a time-of-flight sensor (e.g., a time-of-flight camera). Forexample, the device emits an outgoing beam then measures the phase shiftof the beam at the sensor (receiver).

The apparatus identifies (608) one or more areas of interest in thescene in accordance with variation of depth in the scene as detected atthe first resolution (e.g., using area of interest sub-module 241, FIG.2). In some embodiments, areas of interest in the scene are identifiedin accordance with identification of depth disrupting edges. Forexample, FIG. 4A shows areas of interest 402 identified in scene 300using their depth disrupting edges. Area of interest 402-1 includesdepth disrupting edges between hand 302 and couch 306 and area ofinterest 402-2 includes depth disrupting edges between lamp 304 andcouch 306. In some embodiments, the one or more areas of interest in thescene are identified in accordance with a focus of the apparatus (e.g.,a focus of a camera on the apparatus). In some instances, the focus isauto-set and, in other instances, the focus set by a user of theapparatus.

In some embodiments, the apparatus uses (610) the first depth map toselect the one or more areas of interest (e.g., using area of interestsub-module 241, FIG. 2) and the first depth map has the firstresolution. For example, the apparatus identifies (612) edges within thescene using the first depth map (e.g., using edge detection sub-module240, FIG. 2) and identifies the one or more areas of interestaccordingly. Areas of interest 402 in FIG. 4A may be identified using adepth map generated by capturing an image of scene 300 illuminated withprojected pattern 310 in FIG. 3B. In some embodiments, the one or moreareas of interest include the entire scene.

For each area of interest (614), the apparatus applies (616), via aprojector (e.g., projector 104, FIG. 2), a respective structured-lightpattern to the area of interest (e.g., using projection module 230, FIG.2). For example, FIG. 4A shows areas of interest 402, FIG. 4C showsstructured-light pattern 404 applied to area of interest 402-1, and FIG.5B shows structured-light patterns 502-1 and 502-2 applied to area ofinterest 402-2.

In some embodiments, to apply a structured-light pattern to a respectivearea of interest, the apparatus generates (618) the respectivestructured-light pattern based on one or more features of the respectivearea of interest (e.g., using pattern module 232, FIG. 2). FIG. 5B showsprojected patterns 502 generated such that lines within each pattern areprojected across depth disrupting edges (e.g., the edges of lamp 304)with a specified orientation (e.g., such that at least some linesintersect an object's edge from one side at a specified angle).

For example, the apparatus generates (620) one or more lines of lightperpendicular to a detected edge within the respective area of interest(e.g., using pattern module 232, FIG. 2). Thus, in some embodiments,generating the respective structured-light pattern based on the one ormore features of the respective area of interest includes generating oneor more lines of light perpendicular to a detected edge within therespective area of interest. FIG. 5B shows projected patterns 502generated such that lines within each pattern are perpendicular to thedetected edges (e.g., the edges of lamp 304).

In some embodiments, the respective structured-light pattern includes(622) a line pattern. For example, patterns 502 in FIG. 5B include linepatterns. In some embodiments, the respective line pattern includes agrid of perpendicular lines. For example, pattern 502-1 in FIG. 5Bincludes a grid of perpendicular lines. In some embodiments, therespective structured-light pattern includes dots, lines, and/or othershapes of structured light.

In some embodiments, the apparatus adjusts (624) a focus of theprojector to focus the respective structured-light pattern on therespective area of interest (e.g., using projection module 230 and/orfocus module 238, FIG. 2). For example, projector 104 shown in FIG. 1focuses on area of interest 402-2 as shown in FIG. 5B, such thatpatterns 502 are primarily applied within the area of interest. In someembodiments, the apparatus adjusts a focus of a camera (e.g., camera102, FIG. 2) to focus on the respective area of interest.

For each area of interest (614, FIG. 6B), the apparatus captures (626),via a camera (e.g., camera 102, FIG. 2), an image of the area ofinterest with the respective structured-light pattern applied to it(e.g., using imaging module 234, FIG. 2). For example, the cameracaptures area of interest 402-2 with patterns 502 applied as shown inFIG. 5B. In some embodiments, the projector projects thestructured-light pattern at a wavelength not generally visible to thehuman eye (e.g., infrared wavelengths) and the camera is configured tocapture the projected structured-light patterns at the givenwavelength(s). For example, the camera is an infrared camera and theprojector uses infrared lasers or LEDs.

In some embodiments, the apparatus senses (628) the line pattern asapplied to the respective area of interest (e.g., using imaging module234, FIG. 2) and uses distortions in the sensed line pattern todetermine depth (e.g., using depth module 236, FIG. 2). Thus, in someembodiments, capturing (626) the image includes sensing the line patternas applied to the respective area of interest; and creating (630, below)the respective depth map for the respective area of interest includesusing distortions in the sensed line pattern to determine depth. Forexample, device 100 senses pattern 404 applied to area of interest 402-1as shown in FIG. 4C and uses the distortions of lines within sensedpattern 404 to determine depth.

For each area of interest (614), the apparatus creates (630) arespective depth map of the area of interest using the image of the areaof interest with the respective structured-light pattern applied to it(e.g., using depth module 2326 FIG. 2). The respective depth map has ahigher resolution than the first resolution. For example, device 100generates a respective depth map for area of interest 402-1 using animage of area of interest 402-1 with pattern 404 applied, as shown inFIG. 4C, and/or generates a respective depth map for area of interest402-2 using an image of area of interest 402-2 with patterns 502-1 and502-2 applied, as shown in FIG. 5B. In some embodiments, the respectivedepth map for each area of interest has a higher resolution than thefirst depth map.

In some embodiments, the apparatus performs (632) two or more iterationsof the applying (616), the capturing (626), and the creating (630) togenerate one or more additional depth maps of a particular area ofinterest (or of each area of interest or a subset thereof). In addition,the apparatus varies the respective structured-light pattern applied tothe particular area of interest for each iteration of the two or moreiterations (e.g., using pattern module 232 and/or projection module 230,FIG. 2). In some embodiments, the apparatus compares and/or combines theadditional depth maps to generate a more precise depth map for theparticular area of interest. In some embodiments, the apparatus: (1)applies multiple structured-light patterns to a particular area, (2)obtains multiple depth maps, and (3) compares and/or combines the depthmaps to increase the depth precision for the particular area. In someembodiments, a particular structured-light pattern is adjusted (e.g.,rotated 5-10 degrees) for each iteration. For example, FIG. 4C showsarea of interest 402-1 with pattern 404 applied and FIG. 4D shows areaof interest 402-2 with pattern 406 applied. Pattern 406 is generated byrotating pattern 404. In some embodiments, the iterations are continueduntil particular projected lines within the structured-light pattern aredetermined to be perpendicular to respective depth disrupting edgeswithin the area of interest.

In some embodiments, the apparatus adjusts the structured light patternby varying (634) an angle at which parallel lines of light of thestructured-light pattern intersect a detected edge within the particulararea of interest (e.g., using pattern module 232, FIG. 2), such that theangle is distinct in each iteration. For example, FIG. 4C shows area ofinterest 402-1 with pattern 404 applied such that the lines of pattern404 intersect the edges of hand 302 at a first angle; and FIG. 4D showsarea of interest 402-2 with pattern 406 applied such that the lines ofpattern 406 intersect the edges of hand 302 a second angle, distinctfrom the first angle.

In some embodiments, the apparatus combines (636) the respective depthmap for each area of interest of the one or more areas of interest withthe first depth map to obtain a detailed depth map of the scene (e.g.,using depth module 236, FIG. 2). In some embodiments, at least a portionof the detailed depth has a higher resolution than the first depth map(e.g., is more precise), since each respective depth map has a higherresolution than the first depth map. In some embodiments, the apparatuscombines the respective depth map for each of at least a subset of theone or more areas of interest with the first depth map to obtain adetailed depth map of the scene. In some embodiments, the apparatussends the respective depth maps to an external device (e.g., viacommunication interface(s) 204, FIG. 2) and receives the detailed depthmap from the external device (e.g., via communication interface(s) 204).

In some embodiments, the apparatus captures (638) an image of the scene(e.g., using imaging module 234 in conjunction with camera 102, FIG. 2)with a focus adjusted in accordance with the detailed depth map (e.g.,adjusted using focus module 238, FIG. 2). Alternatively, the focus isadjusted in accordance with a respective depth map for an area ofinterest. Thus, in some embodiments, the apparatus adjusts a focus ofthe camera in accordance with the detailed depth map; and captures animage of the scene with the adjusted focus.

In some embodiments, the apparatus identifies (640) one or more objectsin the scene (e.g., using object recognition module 244, FIG. 2) usingthe detailed depth map. For example, device 100 identifies hand 302,lamp 304, and/or couch 306 in scene 300 as shown in FIG. 3A.

In some embodiments, the apparatus enables (642) three-dimensional (3-D)gesture control (e.g., using 3-D module 246, FIG. 2) on the apparatususing the detailed depth map. In some embodiments, the apparatus usesthe detailed depth map to improve 3-D gesture control. For example, theapparatus uses the detailed depth map to enable more accurate gesturerecognition than otherwise available.

Attention is now directed to selective flash illumination, embodimentsof which are described with respect to FIGS. 7A-10B.

FIGS. 7A-7D are diagrams illustrating exemplary implementations of aflash array. FIGS. 7A-7C show top-down views of various flash arrays.FIG. 7A shows flash array 700 including flash elements 702, 704, 706,708, and 709 with varying sizes. FIG. 7B shows flash array 710 includingflash elements 712, 714, 716, 718, 720, 722, 724, 726, 728, and 730.FIG. 7B further shows flash elements with varying shapes such as flashelement 712 (rounded rectangle) and flash element 726 (circle).(Alternatively, the flash elements in an array may be of identicalshapes and/or sizes.) In some embodiments, the flash elements of flasharray 700 and/or flash array 710 are LEDs (e.g., OLEDs) while in otherembodiments the flash elements, or a subset of the flash elements, areother optical devices such as flashbulbs or air-gap flash devices.

FIG. 7C shows flash array 750 including LEDs 752 (e.g., LED 752-1through 752-15). In some embodiments, individual LEDs 752 are the samesize and shape as one another (as shown in FIG. 7C) while in otherembodiments individual LEDs 752 vary in size and/or shape (as shown inFIG. 7B). In some embodiments, LEDs 752 are OLEDs.

FIG. 7D shows a side-view of flash device 760. Flash device 760 includeslens 764, flash elements 762, and control circuitry 760. In someembodiments, lens 764 is a Fresnel lens while in other embodiments othertypes of lenses are used. In some embodiments, various subsets of flashelements 762 are coupled to distinct lenses. For example, each flashelement 762 is coupled to a distinct Fresnel lens. In some embodiments,multiple distinct types of Fresnel lenses are used. In some embodiments,flash elements 762 are LEDs. In some embodiments, control circuitry 760includes, or is coupled to, a substrate. In some embodiments, controlcircuitry 760 is coupled to one or more processors (e.g., processor(s)202, FIG. 2) and/or memory (e.g., memory 206, FIG. 2). In someembodiments, control circuitry 760 includes one or more controllers 254(FIG. 2) (e.g., one or more processors). In some embodiments, controlcircuitry 760 is configured to supply distinct power levels to each ofthe elements 762 (or to respective groups of elements 762), such thatthe power levels provided to each element 762 (or to each group ofelements 762) is independently controlled. For example, controlcircuitry 760 is configured to supply an individually controllable, andthus potentially distinct, amount of electrical current to each of theelements 762. In some embodiments, control circuitry 760 is governed bya flash module (e.g., flash module 248, FIG. 2).

In some embodiments, flash 108 (FIGS. 1B and 2) includes at least oneof: flash array 700 (FIG. 7A), flash array 710 (FIG. 7B), flash array750 (FIG. 7C) and flash device 760 (FIG. 7D). The flash arrays 700, 710,and 750 are examples of flash arrays 252 (FIG. 2).

FIG. 8 illustrates exemplary direction of flash illumination towardareas of interest in a scene. FIG. 8 shows scene 300 includingidentified areas of interest 402-1 and 402-2. FIG. 8 further shows flasharray 750 including LEDs 752. In the example of FIG. 8, flashillumination is directed toward areas of interest 402-1 and 402-2 byvarying the power levels supplied to LEDs 752. In this example, LED752-6 and 752-10 are supplied with the highest power level, denoted bycross-hashed patterns; LEDs 752-1, 752-5, 752-7, 752-9, 752-11, and752-15 are supplied with an intermediate power level, denoted by linedpatterns; and remaining LEDs 752 are supplied with a lower power. Insome embodiments, other LEDs 752 are supplied with the highest powerlevel and/or the intermediate power level. For example, LED 752-2 issupplied with the intermediate power level. In some embodiments, thelowest power level represents no power being supplied (e.g., thecorresponding LEDs are off) while in other embodiments, each LED 752 issupplied with some power (e.g., is on). In some embodiments, more thanthree power levels are supplied. For example, LED 752-7 is supplied witha second intermediate power level, distinct from the power levelsupplied to LED 752-1. In some embodiments, each LED 752 is suppliedwith a distinct power level. Thus, in accordance with some embodiments,flash array 750 is supplied with fifteen distinct power levels (e.g.,one power level for each LED in the array).

FIGS. 9A-9C illustrate exemplary flash optimization for directing flashillumination toward a moving area of interest in a scene. FIG. 9A showsscene 300 with identified area of interest 902 at position 902-a. FIG.9A further shows flash array 710 including flash elements 712 through730. In the example of FIG. 9A, flash illumination is directed towardarea of interest 902 (e.g., the palm of hand 302) by supplying morepower to flash element 714 than to the other flash elements in flasharray 710. In some embodiments, other flash elements adjacent to flashelement 714 (e.g., flash elements 720, 724, 728, 712, and/or 718) aresupplied more power than non-adjacent flash elements (e.g., flashelement 730). In some embodiments, all flash elements other than flashelement 714, or other than flash element 714 and one or more adjacentflash elements, are not supplied power and thus do not produce light.

FIG. 9B shows area of interest 902 moving to position 902-b. FIG. 9Bfurther shows flash array 710 directing illumination toward area ofinterest 902 by supplying a high power level to flash element 728,denoted by the cross-hashed pattern; supplying an intermediate powerlevel to flash element 718, denoted by the lined pattern; and supplyinga low power level (which may be no power) to the remaining flashelements. In accordance with some embodiments, a device (e.g., device150, FIG. 2) predicts the movement of area of interest 902 (e.g., fromposition 902-a to position 902-b) with a motion prediction module (e.g.,motion prediction module 250, FIG. 2) and directs flash illuminationbased on the predicted movement.

FIG. 9C shows area of interest 902 moving to position 902-c. FIG. 9Cfurther shows flash array 710 directing illumination toward area ofinterest 902 by supplying a high power level to flash elements 716 and718, denoted by the cross-hashed pattern; and supplying a low powerlevel (which may be no power) to the remaining flash elements.

FIGS. 10A-10B are flow diagrams illustrating exemplary method 1000 ofgenerating and using directed flash illumination, in accordance withsome embodiments. In some embodiments, method 1000 is governed byinstructions that are stored in a non-transitory computer-readablestorage medium (e.g., in memory 206, FIG. 2) and that are executed byone or more processors of an apparatus (e.g., processor(s) 202 of device150 and/or controller 254 of flash 108, FIG. 2). In some embodiments,method 1000 is performed at a device (e.g., device 150) comprising acamera, one or more processors, and memory storing one or more programsconfigured for execution by the one or more processors. As describedbelow, method 1000 provides an efficient and precise way to direct flashillumination toward areas of interest in a scene.

In some embodiments, prior to directing flash illumination, a deviceidentifies (1002) an area of interest (or multiple areas of interest) inthe scene (e.g., with area of interest sub-module 241, FIG. 2) based onuser input specifying the area(s) of interest. In some embodiments, theuser selects a focus target for a camera on the device and the focustarget is identified as the area of interest. In some embodiments, theuser selects a flash target as the area of interest. In accordance withsome embodiments, area of interest 902 in FIG. 9A is identified based onuser input (e.g., via user interface 210, FIG. 2). In some embodiments,the device will disregard an area of interest selected by the user(e.g., if the objects in the area of interest are outside of anilluminated capture area for the device). In some embodiments, thedevice generates an error in accordance with a determination that theuser has specified an area of interest containing objects outside of theilluminated capture area (e.g., displays an error message to the user).

In some embodiments, prior to directing flash illumination, the deviceidentifies (1004) an area of interest in a scene (e.g., with area ofinterest sub-module 241, FIG. 2) using a depth map. In some embodiments,the device generates the depth map (e.g., with depth module 236, FIG.2). In some embodiments, the device obtains the depth map from anexternal source (e.g., via communication interface(s) 204, FIG. 2). Inaccordance with some embodiments, multiple areas of interest (e.g.,areas of interest 402, FIG. 8) are identified using a depth map. In someembodiments, the device uses the depth map to determine which objects inthe scene are within an illuminated capture area (e.g., illuminatedcapture area 110, FIG. 1B).

In some embodiments, prior to directing flash illumination, the deviceidentifies (1006) an area of interest in the scene (e.g., with area ofinterest sub-module 241, FIG. 2) in accordance with light metering. Insome embodiments, the device includes one or more light meteringcomponents. In some embodiments, the device obtains light meteringinformation from an external source (e.g., via communicationinterface(s) 204, FIG. 2). In accordance with some embodiments, multipleareas of interest (e.g., areas of interest 402, FIG. 8) are identifiedin accordance with light metering.

In some embodiments, prior to directing flash illumination, the deviceidentifies (1008) an area of interest (or multiple areas of interest) ina scene (e.g., with area of interest sub-module 241, FIG. 2) inaccordance with object detection. In some embodiments, the devicedetects objects in the scene (e.g., with object recognition module 244,FIG. 2) while in other embodiments, the device obtains object detectioninformation from an external source (e.g., via communicationinterface(s) 204, FIG. 2). In accordance with some embodiments, area ofinterest 902 in FIGS. 9A-9C is identified in accordance with objectdetection (e.g., the detection of hand 302). In some embodiments, thedevice identifies the area of interest in the scene in accordance withfacial detection, such that the area of interest is a face.

In some embodiments, the device identifies (1010) an object in apreviously captured image of the scene (e.g., with object recognitionmodule 244, FIG. 2); and selects an area containing the identifiedobject as the area of interest (e.g., with area of interest sub-module241, FIG. 2). Thus, in some embodiments: (1) the image of operation 1028(below) is a first image, and (2) identifying the area of interest inthe scene in accordance with object detection includes: (a) identifyingan object in a second image of the scene that was captured before thefirst image; and (b) selecting an area containing the identified objectas the area of interest. For example, (1) FIG. 9A illustrates the firstimage; (2) FIG. 3A illustrates a previously captured image of scene 300;and (3) the device identifies hand 302 in FIG. 3A and accordinglyselects area of interest 902 in FIG. 9A.

The device directs (1012) flash illumination to the identified area(s)of interest in the scene (e.g., with flash module 248, FIG. 2). Forexample, FIG. 9A shows area of interest 902 and flash array 710directing flash illumination toward area of interest 902 (e.g., bysupplying a higher power level to flash element 714 than to the otherflash elements on the array). In some embodiments, prior to directingflash illumination, the device determines that identified area ofinterest is within an illuminated capture area for the flash (e.g.,illuminated capture area 110, FIG. 1B).

To direct the flash illumination to the identified area(s) of interest,the device adjusts (1014) power supplied to respective elements of aflash array (e.g., with power supply sub-module 249, FIG. 2). Forexample, FIG. 8 shows areas of interest 402 and flash array 750directing flash illumination toward the areas of interest by supplying ahigh power level to LEDs 752-6 and 752-10, supplying an intermediatepower level to LEDs 752-1, 752-5, 752-7, 752-9, 752-11, and 752-15, andsupplying a lower power level to the remaining LEDs on the array. Insome embodiments, the device adjusts the power supplied to therespective elements based in part on distances between the device andobjects in the scene (e.g., objects within the identified area ofinterest).

In some embodiments, the flash array (e.g., flash 108, FIG. 1B) includes(1016) a plurality of LEDs. For example, FIG. 7C shows flash array 750including LEDs 752.

In some embodiments, the device provides (1018) distinct (orindividually controlled) power levels to respective LEDs of theplurality of LEDs (e.g., with power supply sub-module 249, FIG. 2).Thus, in some embodiments, the adjusting (1014) includes providingdistinct power levels to respective LEDs of the plurality of LEDs. Forexample, FIG. 8 shows three distinct power levels provided to LEDs 752:a high power level to LEDs 752-6 and 752-10; an intermediate power levelto LEDs 752-1, 752-5, 752-7, 752-9, 752-11, and 752-15; and a lowerpower level to the remaining LEDs on the array.

In some embodiments, the device provides (1020) a distinct power levelto each LED of the plurality of LEDs (e.g., with power supply sub-module249, FIG. 2). Thus, in some embodiments, the providing (1014) includesproviding a distinct power level to each LED of the plurality of LEDs.For example, each LED 752 on flash array 750 in FIG. 8 is provided adistinct power level.

In some embodiments, the plurality of LEDs includes (1022) atwo-dimensional (2-D) array of LEDs. In some embodiments, the 2-D arrayof LEDs forms a rectangle, as shown in FIG. 7A, while in otherembodiments, the 2-D array forms another shape such as arounded-rectangle, a cross, or an oval. In various embodiments, thearray of LEDs forms or is situated on a concave or convex surface orother curved surface. In some embodiments, the flash illuminationgenerated by the LEDs is shaped by one or more lenses. In someembodiments, the device further includes a lens covering the flasharray, as shown in FIG. 7D, and the lens shapes illumination generatedby the flash array.

In some embodiments, the plurality of LEDs includes (1024) two or moreLEDs of distinct sizes. For example, FIG. 7A shows flash array 700including flash elements (e.g., LEDs) of various sizes. In someembodiments, the plurality of LEDs includes two or more LEDs of distinctshapes. For example FIG. 7B shows flash array 710 including flashelements (e.g., LEDs) of various shapes.

In some embodiments: the device supplies (1026) power to only a subsetof the flash array (e.g., with power supply sub-module 249, FIG. 2) thatis positioned to concentrate the flash illumination on the area ofinterest in the scene. For example, the device in FIG. 9A supplies powerto only flash element 714 in order to concentrate the flash illuminationon area of interest 902. In some embodiments, power is supplied to theentire array and additional power is supplied to a subset of the array(e.g., for additional illumination).

The device captures (1028), via a camera (e.g., camera 102, FIG. 1B), animage of the scene as illuminated by the flash illumination. Forexample, the device in FIG. 9A captures scene 300 as illuminated byflash array 710. In some embodiments, the device captures the image witha camera (e.g., camera 102, FIG. 2) in conjunction with an imagingmodule (e.g., imaging module 234, FIG. 2). In some embodiments, thedevice captures video images of the scene as illuminated by flashillumination. For example, a device captures video images of a scene asilluminated by flash array 710 as illustrated in FIGS. 9A-9C.

In some embodiments, the device predicts (1030) subsequent movement ofan object in the scene (e.g., with motion prediction module 250, FIG.2). In accordance with some embodiments, the device capturing the scenein FIGS. 9A-9C predicts motion of area of interest 902, corresponding tomovement of hand 302, from position 902-a (FIG. 9A) to position 902-b(FIG. 9B) to position 902-c (FIG. 9C).

In some embodiments, the device redirects (1032) flash illumination tothe object (e.g., with flash module 248, FIG. 2) based on the predictedmovement of the object. In the example of FIGS. 9A-9C, the deviceredirects flash illumination from position 902-a (FIG. 9A) to position902-b (FIG. 9B) to position 902-c (FIG. 9C). In some embodiments, thedevice determines whether the predicted movement of the object causesthe object to leave an illuminated capture area for the device (e.g.,illuminated capture area 110, FIG. 1B). In some embodiments, the deviceforgoes redirecting the flash illumination to the object in accordancewith a determination that the object has left the illuminated capturearea for the device. In some embodiments, the device generates an errorin accordance with a determination that the object has left theilluminated capture area (e.g., displays an error message to a user ofthe device).

In some embodiments, the device repeatedly varies (1034) the powersupplied to the respective elements of the flash array (e.g., with powersupply sub-module 249, FIG. 2) to track the predicted movement. In theexample of FIGS. 9A-9C, the device redirects the flash illumination byvarying the power levels supplied to particular flash elements of flasharray 710.

In some embodiments, the device captures (1036) one or more additionalimages as the flash illumination is redirected while the object moves.For example, an image is captured corresponding to each of FIGS. 9A-9Cas illuminated by flash array 710.

Although some of various drawings illustrate a number of logical stagesin a particular order, stages which are not order dependent may bereordered and other stages may be combined or broken out. While somereordering or other groupings are specifically mentioned, others will beapparent to those of ordinary skill in the art, so the ordering andgroupings presented herein are not an exhaustive list of alternatives.Moreover, it should be recognized that the stages could be implementedin hardware, firmware, software or any combination thereof.

The foregoing description, for purpose of explanation, has beendescribed with reference to specific embodiments. However, theillustrative discussions above are not intended to be exhaustive or tolimit the scope of the claims to the precise forms disclosed. Manymodifications and variations are possible in view of the aboveteachings. The embodiments were chosen in order to best explain theprinciples underlying the claims and their practical applications, tothereby enable others skilled in the art to best use the embodimentswith various modifications as are suited to the particular usescontemplated.

What is claimed is:
 1. A method comprising: at a device comprising acamera, one or more processors, and memory storing one or more programsconfigured for execution by the one or more processors: constructing adepth map of an area of interest in a scene according to one or moreimages captured while applying a structured light pattern to the area ofinterest, including: applying multiple, iterative structured lightpatterns to the area of interest to obtain multiple respective depthmaps, wherein: the area of interest in the scene comprises an object;applying the structured light pattern comprises applying one or morelines of the structured light pattern with a specified orientationacross an edge of the object so that a subset of the one or more linesof the structured light pattern intersect the edge of the object from atleast one side at a specified angle; and the structured light pattern isrotated for each iteration until the lines of the structured lightpattern are determined to be perpendicular to the edge of the object;directing, by the device, flash illumination to the area of interest inthe scene, the directing comprising adjusting power supplied torespective elements of a flash array; capturing, via the camera, animage of the scene as illuminated by the flash illumination; predicting,by the device, subsequent movement of the object; redirecting, by thedevice, flash illumination to the object based on the predicted movementof the object; and capturing, via the camera, one or more additionalimages as the flash illumination is redirected while the object moves.2. The method of claim 1, wherein directing the flash illumination tothe area of interest in the scene comprises supplying power to only asubset of the flash array, wherein the subset of the flash array ispositioned to concentrate the flash illumination on the area of interestin the scene.
 3. The method of claim 1, wherein the device furthercomprises the flash array.
 4. The method of claim 1, wherein: the flasharray comprises a plurality of light-emitting diodes (LEDs); and theadjusting comprises providing distinct power levels to respective LEDsof the plurality of LEDs.
 5. The method of claim 4, wherein theproviding comprises providing a distinct power level to each LED of theplurality of LEDs.
 6. The method of claim 4, wherein the plurality ofLEDs comprises a two- dimensional array of LEDs.
 7. The method of claim4, wherein the plurality of LEDs comprises two or more LEDs of distinctsizes.
 8. The method of claim 1, wherein redirecting the flashillumination comprises repeatedly varying the power supplied to therespective elements of the flash array to track the predicted movement.9. The method of claim 1, wherein the structured light pattern isgenerated so that one or more of the lines are perpendicular to the edgeof the object.
 10. A device comprising: a flash array comprising aplurality of light sources; and a controller configured to: construct adepth map of an area of interest in a scene according to one or moreimages captured while applying a structured light pattern to the area ofinterest, including: applying multiple, iterative structured lightpatterns to the area of interest to obtain multiple respective depthmaps, wherein: the area of interest in the scene comprises an object;applying the structured light pattern comprises applying one or morelines of the structured light pattern with a specified orientationacross an edge of the object so that a subset of the one or more linesof the structured light pattern intersect the edge of the object from atleast one side at a specified angle; and the structured light pattern isrotated for each iteration until the lines of the structured lightpattern are determined to be perpendicular to the edge of the object;and adjust power supplied to respective light sources of the pluralityof light sources, to direct flash illumination for a camera to the areaof interest in the scene, wherein the controller is configured topredict subsequent movement of the object and redirect flashillumination to the object based on the predicted movement of theobject.
 11. The device of claim 10, wherein the flash array is disposedin the camera.
 12. The device of claim 10, wherein: the plurality oflight sources comprises a plurality of LEDs; and the adjusting comprisesproviding distinct power levels to respective LEDs of the plurality ofLEDs.
 13. The device of claim 10, wherein redirecting the flashillumination comprises repeatedly varying the power supplied to therespective light sources to track the predicted movement.
 14. The deviceof claim 10, wherein the structured light pattern is generated so thatone or more of the lines are perpendicular to the edge of the object.15. A non-transitory computer-readable storage medium storing one ormore programs for execution by one or more processors, the one or moreprograms comprising instructions for: constructing a depth map of anarea of interest in a scene according to one or more images capturedwhile applying a structured light pattern to the area of interest,including: applying multiple, iterative structured light patterns to thearea of interest to obtain multiple respective depth maps, wherein: thearea of interest in the scene comprises an object; applying thestructured light pattern comprises applying one or more lines of thestructured light pattern with a specified orientation across an edge ofthe object so that a subset of the one or more lines of the structuredlight pattern intersect the edge of the object from at least one side ata specified angle; and the structured light pattern is rotated for eachiteration until the lines of the structured light pattern are determinedto be perpendicular to the edge of the object; directing flashillumination to the area of interest in the scene, the directingcomprising adjusting power supplied to respective elements of a flasharray; capturing an image of the scene as illuminated by the flashillumination; predicting subsequent movement of the object; redirectingflash illumination to the object based on the predicted movement of theobject; and capturing one or more additional images as the flashillumination is redirected while the object moves.
 16. Thenon-transitory computer-readable storage medium of claim 15, whereinredirecting the flash illumination comprises repeatedly varying thepower supplied to the respective elements of the flash array to trackthe predicted movement.
 17. The non-transitory computer-readable storagemedium of claim 15, wherein the structured light pattern is generated sothat one or more of the lines are perpendicular to the edge of theobject.